Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support array_union scalar expr #1362

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from
Draft

Support array_union scalar expr #1362

wants to merge 2 commits into from

Conversation

dharanad
Copy link
Contributor

@dharanad dharanad commented Jan 31, 2025 •
edited
Loading

Which issue does this PR close?

Part of #1042

Rationale for this change

What changes are included in this PR?

How are these changes tested?

@dharanad
impl array_union
059c037 
Signed-off-by: Dharan Aditya <[email protected]>
@dharanad dharanad marked this pull request as ready for review January 31, 2025 13:42
@dharanad dharanad marked this pull request as draft January 31, 2025 15:56
@dharanad
Copy link
Contributor Author

dharanad commented Jan 31, 2025 •
edited
Loading

There is this one case which i haven't added.

          checkSparkAnswerAndOperator(
            sql("SELECT array_union(array(_2 * 1), array(_9, _10)) from t1"))

Because it was failing due to mismatch in ordering between spark and df
Example

  == Results ==
  !== Correct Answer - 1000 ==                                        == Spark Answer - 1000 ==
   struct<array_union(array((_2 * -1)), array(_9, _10)):array<int>>   struct<array_union(array((_2 * -1)), array(_9, _10)):array<int>>
  ![ArraySeq(128, -128, -896)]                                        [ArraySeq(128, -896, -128)]

I am not sure how to deal with this, open to suggestions

@dharanad
update ut
d5858a4 
Signed-off-by: Dharan Aditya <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@dharanad